Field of the Invention
Embodiments of the present invention generally relate to use of delayed duplicate I-pictures in video coding.
Description of the Related Art
Many network-based video applications, e.g., video conferencing, online gaming, remote desktops, broadcasting, wireless displays, video signage, etc., send and/or receive video data with the expectation of near real-time communication and/or operate in bandwidth constrained environments. Further, such applications may rely on periodic transmission of an intra-predicted picture (I-picture) or an instantaneous decoder refresh (IDR). Such transmissions are used, for example, to keep any end-points refreshed, to stop error propagation due to network packet loss, to assist in recovery when there is packet loss and error concealment was not able to recover, or to negate the effect of a scene change. An I-picture is typically 7× the size of a predicted picture (P-picture).
For example, in many video conferencing applications, an intra-predicted picture (I-picture) or an instantaneous decoder refresh (IDR) is transmitted, at a minimum, once every five minutes (or every 9000 pictures). Some video conferencing systems operate with constrained and rigid network bandwidth availability, either because of leased-line restrictions or quality-of-service (QoS) restrictions within a company network. In such constrained network bandwidth conditions, the periodic IDR transmissions cause sudden spikes in network traffic, which may lead to packet loss and indirectly to bad video quality due to network congestion. FIG. 1 is a graph illustrating the IDR instantaneous bandwidth requirement for typical video sequences in a constant bit-rate control network environment. The sharp peaks in this graph correspond to the IDRs. Further, the large size of the IDRs may also cause increased end to end delay which may result in frame skipping by the decoder and lower quality display. Similar issues may occur in other video-based networked applications.
When there is packet loss, and when a concealment algorithm cannot recover the picture, there are various ways available to recover. One way is to send another IDR, which consumes more bandwidth in an already congested network. Another way is to send a P-picture which may refer to a long term reference picture (LTRP) that is error-free reconstructed on the decoder side. However, the LTRP may be distant in time from the current picture, which will lead to transmission of a larger P-picture. Another way is to send an IDR but reduce the encoded size. This may cause flicker in the display because of the reduced quality of the IDR as compared to the P-pictures.
There are two techniques commonly used to help reduce the IDR bandwidth requirement: adaptive intra refresh and gradual decoder refresh. In each of these techniques, differing sets of coding blocks of video data, e.g., macroblocks, are forced to be intra-predicted over a series of N P-pictures such that at the end of the N pictures, all coding blocks have been refreshed. This distribution of the intra refreshing over a group of P-pictures avoids the instantaneous bit rate peaks of IDR. However, in adaptive intra refresh, there is a possibility that refreshed coding blocks will refer to not yet refreshed coding blocks, leading to error propagation. In gradual decoder refresh, refreshed coding blocks are restricted from referring to non-refreshed coding blocks by restricting motion vectors and spatial prediction, which may cause increased bit-rate and lower video quality. Accordingly, other techniques to for decoder refresh are desirable.